Crop management method and apparatus with autonomous vehicles

ABSTRACT

In some embodiments, a method for managing growing crops in a crop growing farm includes operating one or more unmanned aerial vehicles (UAV) to fly over a plurality of sections of a crop growing farm. The UAVs are fitted with a plurality of cameras equipped to generate images in a plurality of spectrums. The plurality of sections of the crop growing farm grow crops of one or more types or varietals. The method further includes taking a plurality of aerial images of the sections of the vineyard in the plurality of spectrums, using the plurality of cameras, while the UAVs are flying over the plurality of sections of the crop growing farm, and executing an analyzer on a computing system to machine analyze the plurality of aerial images for anomalies associated with growing the crops of the one or more types or varietals. The machine analysis takes into consideration topological information of the crop growing farm, as well as current planting information of the crop growing farm.

RELATED APPLICATION

This application is a Continuation-In-Part application of U.S. patentapplication Ser. No. 16/128,309, entitled “Vine Growing ManagementMethod and Apparatus With Autonomous Vehicles”, filed on Sep. 11, 2018,and claims priority to U.S. application Ser. No. 16/128,309, whichSpecification is hereby fully incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to the field of agriculture. Moreparticularly, the present disclosure relates to method and apparatus formanaging growing of crops, e.g., grain, fruit, vegetable, vines of aplurality of varietals, with the assistance of autonomous vehicles.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Unless otherwiseindicated herein, the materials described in this section are not priorart to the claims in this application and are not admitted to be priorart by inclusion in this section.

Agriculture is a major industry in the United States, which is a netexporter of food. As of the 2007 census of agriculture, there were 2.2million farms, covering an area of 922 million acres (3,730,000 km²), anaverage of 418 acres (169 hectares) per farm. Major crops include corn,soybeans, wheat, cotton, tomatoes, potatoes, grapes, oranges, rice,apples, sorghum, lettuce, sugar beets, and so forth.

Agriculture, food, and related industries contributed $1.053 trillion toU.S. gross domestic product (GDP) in 2017, a 5.4-percent share. Theoutput of America's farms contributed $132.8 billion of this sum—about 1percent of GDP. The overall contribution of the agriculture sector toGDP is larger than this because sectors related to agriculture—forestry,fishing, and related activities; food, beverages, and tobacco products;textiles, apparel, and leather products; food and beverage stores; andfood service, eating and drinking places—rely on agricultural inputs inorder to contribute added value to the economy.

However, over the years, farming has become an increasingly difficultbusiness. Today, it is estimated that the American farmer receives just16-cents for every dollar spent on food by the consumer. That is down 50percent from 1980 when the farmers were receiving 31-cents for everydollar spent. Margins, especially on smaller farms, are too thin to haveroom in their operating budgets to purchase new technology andequipment, invest in experimental agricultural practices or adapt to anew environmental and economic climate, and yet continuous innovation isneeded to increase the yields.

For the fruit and vegetable segment, with the increasing interest amongAmericans in healthy living, there has been a steady increase in demandfor fresh fruits and vegetables, including organic fruits andvegetables. The U.S. fruit and vegetable market was valued at USD 104.7billion in 2016. Vegetables and fruits are presently reigning as theU.S. top snacking items. Like farming in general, owning and operatingan orchard or vegetable farm is a tough business. Huge amount ofinvestment in the U.S. is expected in terms of technology to improve theyield and quality of the products, and their efficient transport.

For the wine industry, consumption in America has steadily increased inthe last two decades, growing from about 500 million gallons in the year1996 to about 949 million gallons in 2016¹. The value of the total U.S.Wine Market for the year 2017 is estimated to be $62.7 billion, ofwhich, $41.8 billion are domestically produced². Currently, for the year2018, the number of wineries in U.S. is estimated to be about 9,654³.The total vine growing acres in the U.S. was estimated to exceed1,000,000 acres, as far back as 2012⁴ ¹ Source: Wine Institute, DOC,BW166/Gomberg, Fredrikson & Associates estimates. Preliminary Historyrevised.² Source: Wines & Vines, 2018, BW166, 2018.³ Source: Statisa—TheStatistics Portal.⁴ The world's grape growing (vineyard) surface area2000-2012 by Per Karlsson, Jun. 6, 2013, Winemaking & Viticulture.

Owning and operating a vineyard is a tough business. “To take on thechallenge of running a winery, you need to be determined, fearless, andpassionate about your craft—although owning a vineyard seems romantic,the wine-making business is a tough one.”⁵ In addition to the upfrontfinancial investments required for the land and the infrastructure (likebuilding, bottling and cellar equipment, trucks, and so forth), thereare multitude of potential problems that could arise with growing vines.Examples of these problems may include, but are not limited to, over orunder irrigation, diseases (such as mildews and black rots), or pests(such as berry moth, Japanese beetles, and rose chafers). Further, theseproblems may vary from one varietal to another. And a vineyard typicallygrows vines of multiple varietals. It is not uncommon for a vineyard tospan over 100 acres, with over 1000 vines planted per acre. And multiplevarietals are planted in different sections of the vineyard. ⁵ TheEconomics of Running a Winery, Aug. 20, 2018, Caroline Goldstein.

For the craft beer industry, consumption in America has also steadilyincreased in the recent years, 2018 saw 7,346 operating U.S. craftbreweries in 2018—4,521 microbreweries, 2,594 brewpubs, 231 regionalbreweries. Craft brewers produced 25.9 million barrels of beer. Retaildollar value for craft beer sold in 2018 was $27.6 billion. Resultantly,there has been significant increase in interest in increasing theefficient growing and production of hops.

Similarly, with the enactment of the 2018 Farm Bill on Dec. 20, 2018,removing hemp from schedule I of the Controlled Substances Act, makinghemp no longer a controlled substance, and with increasing number Stateslegalizing the medical and recreational use of marijuana, likewise,there has been significant in increase in interest in increasing theefficient growing and production of cannabis.

Thus, methods and apparatuses that can improve the management of growingcrops of various types are desired.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will be readily understood by the following detaileddescription in conjunction with the accompanying drawings. To facilitatethis description, like reference numerals designate like structuralelements. Embodiments are illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings.

FIG. 1 illustrates an overview of a system for managing growing crops ofthe present disclosure, in accordance with various embodiments.

FIGS. 2A-2C illustrate taking of aerial images and terrestrial images ofcrops in the example vineyard of FIG. 1, with the assistance ofautonomous vehicles, according to various embodiments.

FIG. 3 illustrates a process for managing growing crops with theassistance of autonomous vehicles, and components of the crop growingmanagement system of FIG. 1, according to various embodiments.

FIGS. 4A-4 b illustrate an example visual report to assist in managinggrowing crops, and an example user interface for interactively viewingthe visual report, according to various embodiments.

FIG. 5 illustrates an example process for machine processing the aerialimages taken with the assistance of autonomous vehicles, according tovarious embodiments.

FIG. 6 illustrates an example process for machine analyzing the aerialand/or terrestrial images, according to various embodiments.

FIG. 7 illustrates an example process for machine generating the visualreport to assist in managing growing vines of a number of varietals,according to various embodiments.

FIG. 8 illustrates an example process for machine facilitatinginteractive viewing of the visual report, according to variousembodiments.

FIG. 9 illustrates an example neural network suitable for use withpresent disclosure to analyze aerial and/or terrestrial images for cropgrowing anomalies, according to various embodiments;

FIG. 10 illustrates an example software component view of the cropgrowing management system of FIG. 1, according to various embodiments.

FIG. 11 illustrates an example hardware component view of the hardwareplatform for the crop growing management system of FIG. 1, according tovarious embodiments.

FIG. 12 illustrates a storage medium having instructions for practicingmethods described with references to FIGS. 1-9, according to variousembodiments.

FIGS. 13a-13d illustrate a number of example seasonal NDVI profiles fora number of example crops grown in a number of sections of a number ofexample crop growing farms.

FIG. 14 illustrates a number of example mean seasonal NDVI profiles fora number of example crops grown in a number of sites of an examplegeographical region.

DETAILED DESCRIPTION

To address challenges discussed in the background section, apparatuses,methods and storage medium associated with managing growing crops, suchas vines in a vineyard, are disclosed herein. In some embodiments, amethod for managing growing crops includes operating one or moreunmanned aerial vehicles (UAV) to fly over a plurality of sections of acrop growing farm, such as a vineyard. The UAVs are fitted with aplurality of cameras equipped to generate images in a plurality ofspectrums. The plurality of sections of the crop growing farm may growcrops of various types, e.g., a vegetable farm may grow variousvegetables, an orchard may grow various fruits, a vineyard may growvines of a plurality of varietals, and so forth. The method furtherincludes taking a plurality of aerial images of the sections of the cropgrowing farm in the plurality of spectrums, using the plurality ofcameras, while the UAVs are flying over the plurality of sections of thecrop growing farm; storing the plurality of aerial images of a pluralityof spectrums of the crops of various types being grown, e.g., the vinesof the plurality of varietals being grown, in a computer readablestorage medium (CRSM), and executing an analyzer on a computing systemto machine analyze the plurality of aerial images for anomaliesassociated with growing the crops of various types, e.g., the vines ofthe plurality of varietals. The machine analysis takes intoconsideration topological information of the crop growing farm, e.g.,the vineyard, as well as current planting information of the cropgrowing farm, e.g., the vineyard.

In some embodiments, the method includes operating one or moreterrestrial robots to traverse the plurality of sections of the cropgrowing farm, such as a vineyard. The one or more terrestrial robots arefitted with one or more cameras equipped to generate images in visualspectrum. The method further includes taking a plurality of visualspectrum terrestrial images of the crops of various types, e.g., vinesof a plurality of varietals, being grown in the plurality of sections ofthe crop growing farm, e.g., a vineyard, using the one or more camerasfitted on the one or more terrestrial robots, while the one or moreterrestrial robots are traversing the plurality of sections of the cropgrowing farm, e.g., a vineyard; storing the plurality of visual spectrumterrestrial images of the vines of the plurality of varietals beinggrown, in the CRSM; and executing the analyzer on the computing systemto machine analyze the plurality of visual spectrum terrestrial imagesfor additional anomalies associated with growing the crops of varioustypes, e.g., vines of a plurality of varietals, in the plurality ofsections of the crop growing farm, e.g., a vineyard. The machineanalysis of the visual spectrum terrestrial images takes intoconsideration phenology information of the various types of crops beinggrown, at different growing stages.

These and are other aspects of the methods and apparatuses for managingcrop growing, in particular, managing growing vine in a vineyard will befurther described with references to the Figures. In the followingdetailed description, reference is made to the accompanying drawingswhich form a part hereof wherein like numerals designate like partsthroughout, and in which is shown by way of illustration embodimentsthat may be practiced. It is to be understood that other embodiments maybe utilized and structural or logical changes may be made withoutdeparting from the scope of the present disclosure. Therefore, thefollowing detailed description is not to be taken in a limiting sense,and the scope of embodiments is defined by the appended claims and theirequivalents.

Various operations may be described as multiple discrete actions oroperations in turn, in a manner that is most helpful in understandingthe claimed subject matter. However, the order of description should notbe construed as to imply that these operations are necessarily orderdependent. In particular, these operations may not be performed in theorder of presentation. Operations described may be performed in adifferent order than the described embodiment. Various additionaloperations may be performed and/or described operations may be omittedin additional embodiments.

For the purposes of the present disclosure, the phrase “A and/or B”means (A), (B), or (A and B). For the purposes of the presentdisclosure, the phrase “A, B, and/or C” means (A), (B), (C), (A and B),(A and C), (B and C), or (A, B and C). The description may use thephrases “in an embodiment,” or “In some embodiments,” which may eachrefer to one or more of the same or different embodiments. Furthermore,the terms “comprising,” “including,” “having,” and the like, as usedwith respect to embodiments of the present disclosure, are synonymous.

As used herein, the term “module” may refer to, be part of, or includean Application Specific Integrated Circuit (ASIC), an electroniccircuit, a processor (shared, dedicated, or group) and/or memory(shared, dedicated, or group) that execute one or more software orfirmware programs, a combinational logic circuit, and/or other suitablecomponents that provide the described functionality.

Referring now to FIG. 1, wherein an overview of a system for managinggrowing crops of the present disclosure, in accordance with variousembodiments, is illustrated. For ease of understanding, system 100 willbe described with reference to managing growing vine of a plurality ofvarietals in a vineyard, however, the disclosure is not so limited.System 100 may be employed to manage growing crops of various types invarious crop growing farms. Examples of crops growing that may bemanaged include, but are not limited to, growing of Hops, Kiwis, Apples,Berries, Cherries, Citrus Fruit, Avocado, Pears, Plums, Hemp, Cannabis,and so forth. Examples of crop growing farms include, but are notlimited, vegetable farms, orchards, vineyards, and so forth. In the caseof growing vine, examples of varietals may include, but are not limitedto, cabernet sauvignon, pinot noir, chardonnay, pinot gris, and soforth. In the case of growing cannabis, example of varietals may includecannabis indica or Cannabis sativa species.

As shown, system 100 for managing growing crops in a crop growing farm102, e.g., growing vines in a vineyard, may include one or more UAVs104, and crop growing management system 110 having crop growingmanagement software 120. In some embodiments, system 100 may furtherinclude one or more terrestrial robots 108. Collectively, one or moreUAVs 104 are fitted with a plurality of cameras to capture aerial imagesin a plurality of spectrums. The UAVs 104 are operated to fly over cropgrowing farm, e.g., a vineyard, 102 and capture aerial images of varioussections of crop growing farm, e.g., a vineyard, 102 in the plurality ofspectrums. Different crop types, e.g., fruits or vegetables of differenttypes, fruits of different types, cannabis or vines of differentvarietals, may be grown in different sections of crop growing farm 102.The plurality of spectrums may include, but are not limited to, visual,red, near infrared, near red, and so forth.

Crop growing management software 120 is configured to be executed on oneor more computer processors of crop growing management system 110, tomachine process the aerial images taken, and machine analyze the aerialimages for anomalies associated with growing crops of various types,e.g., over or under irrigation, and generate a visual report withindications of the anomalies. The machine analysis and reporting takesinto consideration topological information of the crop growing farm,e.g., natural or man made geographical features, like a pond, a buildingand so forth, current planting information, e.g., the crop types orvarietals being grown, and where in crop growing farm 102.

For embodiments where system 100 also includes one or more terrestrialrobots 106, each of the one or more terrestrial robots 106 is equippedwith one or more cameras to capture terrestrial images in visualspectrum, in one or more directions. The one or more directions mayinclude a left (up/straight ahead/down) outward looking direction, aright (up/straight ahead/down) outward looking direction, a forward(up/straight ahead/down) outward looking direction, and/or a backward(up/straight ahead/down) outward looking direction. The one or moreterrestrial robots 106 are operated to traverse crop growing farm 102and capture terrestrial images of the crops of various types orvarietals being grown in the various sections of crop growing farm 102,in the visual spectrum. The images of the crops may show various aspectsof the crops, e.g., for vines, the images may show various aspects ofthe vines, e.g., its grapes, its leaves, its roots, and so forth.

Crop growing management software 120 is further configured to beexecuted on the one or more computer processors of crop growingmanagement system 110, to machine process the terrestrial images taken,and machine analyze the terrestrial images of the crops for furtheranomalies associated with growing crops of a particular type orvarietal, e.g., plant diseases and/or pest infestations, and generate avisual report with indications of the anomalies. The machine analysisand reporting additional takes into consideration phenology informationof the crop types or varietals, at different growing stages.

In some embodiments, a plurality of UAVs 104 are employed. Each UAV 104is either successively or concurrently fitted with three (3) cameras forcapturing aerial images in 3 spectrums, visual spectrum, red or nearinfrared spectrum, and near red spectrum. In some embodiments, the 3cameras include a red/green/blue (RGB) camera configured to capture andgenerate images in visual spectrum, a Normalized Difference VegetationIndex (NDVI) camera configured to capture and generate images in red ornear infrared spectrum, and a Normalized Difference Red Edge (NDRE)camera equipped to capture and generate images in near red spectrum. Insome embodiments, one or more UAVs 104 may be fitted with one or moreinfrared thermal cameras to capture aerial thermal images of the varioussections of vineyard 102. The plurality of UAVs 104 are operated tosystematically fly over all sections of vineyard 102, or selectively flyover selected sections of vineyard 102, capturing and generating aerialimages of the all or selected sections of vineyard 102. In someembodiments, the cameras has resolution and zoom in power to allow eachpixel of each aerial image to cover approximately 3-4 cm² of a sectionalarea, with the UAVs 104 operated at 400 ft or below. In alternateembodiments, as camera resolution further improves, each pixel of eachaerial image may cover an area as small as 1 cm².

In some embodiments, a swarm of lightweight UAVs with less FederalAviation Administration (FAA) operation restrictions, such as DragonflyDrone with less height operational restrictions, are employed. ADragonfly Drone is a drone weighing typically less than 1 lb, which isavailable in multiple form factors but less than 9″ in diameter. TheDragonfly Drone may be incorporated with extremely high resolution nearinfrared (NIR), short wave infrared (SWIR), thermal, and/or RGB cameras,amongst a large variety of others. The Dragonfly drones are usedtogether in coordinated swarms mapping together to rapidly take imageryand collect data from the air over large crop growing farms. The smallform factor eliminates the current need to acquire an FAA license to flythe drones. The Dragonfly drones may also contain a magnetic content formagnetic charging, and can return to a WIFI enabled base station forrecharging, uploading of imagery, and downloading of flight plan.

The concurrent employment of the near red spectrum images of the NDREcameras provides certain information that complement the limitations ofthe red or infrared images of the NDVI cameras. The index of NDVI imagesis derived from reflectance values in red and near-infrared bands of theelectromagnetic spectrum. Index values ranging from −1 to 1 indicate theinstantaneous rate of photosynthesis of the crop of interest. NDVI iscommonly thought of as an index of biomass, but a normal NDVI curve willdecline towards the end of the growing season even when the amount ofbiomass is at peak levels. Therefore, NDVI can only be considered anindex of photosynthetic rate, not the amount of foliage. Also, NDVI isnot sensitive to crops with high leaf area index (LAI) and tends tosaturate at high LAI values.

On the other hand, NDRE uses the red-edge portion of the spectrum. Sincered-edge light is not absorbed as strongly as red-light duringphotosynthesis, it can penetrate deeper into the crop canopy and therebysolves the issue of NDVI saturating at high LAI values. NDRE is alsomore sensitive to medium-high chlorophyll levels and can be used to mapvariability in fertilizer requirements or foliar nitrogen demand. Leafchlorophyll and nitrogen levels do not necessarily correlate with soilnitrogen availability and should be ground-truthed with soil or tissuesamples.

In some embodiments, a plurality of terrestrial robots 106 are employed.Each terrestrial robot 106 is fitted with at least two (2) cameras tocapture and generate terrestrial images of the crops of different typesor varietals, in visual spectrum, in at least 2 directions along theroutes traversed by the terrestrial robots 106. Similarly, the pluralityof terrestrial robots 106 are operated to systematically traverse theentire crop growing farm 102 or selectively traverse selected sectionsof crop growing farm 102, capturing and generating terrestrial images ofall crops of all types or varietals or selected crops of selected typesor varietals grown in all or selected sections of crop growing farm 102.

In some embodiments, each UAV 104/robot 106 includes computer readablestorage medium (CRSM) 124/126 to temporarily store theaerial/terrestrial images captured/generated. In some embodiments, theCRSM are removable, such as a universal serial bus (USB) drive, allowingthe captured/generated aerial/terrestrial images to be removablytransferred to CRSM of crop growing management system 110, via acompatible input/output (I/O) port on crop growing management system110. In some embodiments, each UAV 104/robot 106 may include an I/O port(not shown), e.g., a USB port, to allow the stored aerial/terrestrialimages to be accessed and transferred to crop growing management system110. In still other embodiments, each UAV 104/robot 106 may include acommunication interface, e.g., WiFi or Cellular interface, to allow thestored aerial/terrestrial images to be wirelessly transferred to cropgrowing management system 110, via e.g., access point/base station 118.Access point/base station 118 may be communicatively coupled with cropgrowing management system 110 via one or more private or public networks114, including e.g., the Internet.

Except for the cameras fitted to UAVs 104, and the manner UAVs 104 androbots 106 are employed, UAVs 104 and robots 106 may otherwise be anyone of a number of such vehicles/devices known in the art. For example,except for the cameras fitted to UAVs 104, UAVs 104 may be a fixed wingUAV, a tricopter, a qudcopter, a hexcopter or a lightweight UAV. Inparticular, as discussed earlier, UAVs 104 may be lightweight UAVs thatmay operate above 400 ft, Similarly, terrestrial robots 106 may bewheeled, threaded or screw propelled.

Still referring to FIG. 1, in some embodiments, system 100 may furtherinclude various sensors 108 dispositioned at various locations in cropgrowing farm 102 to collect and generate various local environmentaldata. Example of sensors 108 may include, but are not limited to,temperature sensors for sensing local temperature, humidity sensors forsensing local humidity level, moisture sensors for sensing moisturelevel in the soil, and so forth. In various embodiments, sensors 108 maybe equipped to provide the collected sensor data to crop growingmanagement system 110 wirelessly, via access points/base station 118. Insome embodiments, some of these sensors may be dispositioned in UAVs 104and/or terrestrial robots 106 to collect and generate various localenvironmental data as UAVs 104 and/or terrestrial robots 106 fly over ortraverse crop growing farm 102. For these embodiments where sensors 108are employed, crop growing management software 120 further takes theselocal environmental data into consideration, when machine processingaerial images (and optionally, terrestrial images) to machine detectanomalies associated with growing crops of various types or varietals.In some embodiments, sensors 108 may include imaging devices or camerasmounted at various fixtures of the crop growing farm, e.g., utilitypoles, or nettings covering the crops being grown, to capture overheadimages of the corresponding sections of the crop growing farm (tocomplement or supplement the aerial images captured with the UAVs).Examples of netting coverings include netting typically used in farms toreduce wind, sun and bird effects on growing plants. The netting may beincorporated with extremely high resolution near infrared (NIR), shortwave infrared (SWIR), thermal, and/or RGB cameras, amongst a largevariety of others. Multiple cameras can be attached to the underside ofnets, providing both real time and video imagery of plant growth.

In some embodiments, system 100 may further include remote servers 112having repositories of environmental data applicable to crop growingfarm 102, e.g., weather or environmental data services, withtemperature, precipitation, air pollution data for the geographicalregion/area where crop growing farm 102 is located. Crop growingmanagement system 110 may be communicatively coupled with remote servers112 via one or more private or public networks 114, including e.g., theInternet. For these embodiments, crop growing management software 120further takes these regional environmental data into consideration, whenmachine processing and analyzing aerial images (and optionally, overheadand/or terrestrial images) to machine detect anomalies associated withgrowing crops of various types or varietals.

In some embodiments, crop growing management software 120 is furtherconfigured to support interactive viewing of the visual report by auser, via a user computing device 116. User computing device 116 may becoupled with crop growing management system 110 via one or more publicand/or private networks 114, and/or access point/base station 118. Insome embodiments, crop growing management software 120 may facilitateinteractive viewing of the visual report, via web services. For theseembodiments, user computing device 116 may access and interact with thevisual report via a browser on user computing device 116. In someembodiments, crop growing management software 120 may provide an agente.g., an app, to be installed on user computing device 116 to access andinteract with the visual report. Except for its use to access andinteract with the visual report, user computing device 116 may be anyone of a number of computing devices known in the art. Examples of suchcomputing devices may include, but are not limited to, desktopcomputers, mobile phones, laptop computers, computing tablets, and soforth.

Similarly, except for their use to facilitate provision ofaerial/overhead/terrestrial images and/or environment data, as well asaccessing the visual reports, access point/base stations 118 andnetwork(s) 114 may be any one of a number access points/base stationsand/or networks known in the art. Networks 114 may include one or moreprivate and/or public networks, such as the Internet, local area or widearea, having any number of gateways, routers, switches, and so forth.

Before further describing elements of system 100, it should be notedwhile for ease of understanding, UAVs 104, terrestrial robots 106, andcrop growing management system 110 have been described so far, and willcontinue to be mainly described as assisting in management of growingcrops of a plurality of types or varietals in a crop growing farm, thepresent disclosure is not so limited. System 100 may be practiced withUAVs 104, terrestrial robots 106, and crop growing management system 110configured to service multiple crop growing farms growing crops ofmultiple types or varietals.

Referring now to FIGS. 2A-2C, wherein taking of aerial/overhead imagesof a crop growing farm, an example vineyard, and terrestrial images ofcrops being grown, vines growing in the example vineyard with theassistance of autonomous vehicles, according to various embodiments, areillustrated. Shown in FIG. 2A is a composite aerial overhead image 202of a crop growing farm, an example vineyard, 102, generated by combiningor stitching together individual aerial overhead images 204 taken by oneor more UAVs 104, while the one or more UAVs 104 fly over the cropgrowing farm, example vineyard, 102. As described earlier, in someembodiments, the one or more UAVs 104 are operated to systematically flyover all sections of the crop growing farm, example vineyard, 102 at aselected altitude. In various embodiments, each pixel of eachaerial/overhead image covers an area of approximately 3-4 cm² orsmaller. Thus, the number of aerial/overhead images 204 shown in FIG. 2Aas being combined together to cover the crop growing farm, examplevineyard, 102 are significantly less than the number of aerial/overheadimages 204 combined/stitched together in real life. The number isreduced for ease of illustration and understanding, and thus is not tobe read as limiting on the present disclosure.

The different levels of grayness in FIG. 2A correspond to differentcolors in real life in the composite NDVI or NDRE imagescombined/stitched together based on the individual NDVI/NDRE imagestaken, which in turn correspond to different conditions of the soiland/or different conditions of the crops (depending in part, on the croptypes or varietals being grown).

Shown in FIGS. 2B-2C are example terrestrial images 220 and 230 ofcouple of the example vines of couple varietals being grown in couple ofthe sections in example vineyard 102, captured by the camera(s) ofterrestrial robot(s) 106, while terrestrial robot(s) 106 is (are)operated to traverse various sections of example vineyard 102. Asillustrated by the example picture 220 in FIG. 2B, disease (in thiscase, downy mildew), may be machine detected, using the phenologyinformation provide, for various growing stages of the vine varietal. Asillustrated by the example picture 230 in FIG. 2C, infestation (in thiscase, sweetpotato weevil), may be machine detected, using the phenologyinformation provide, for various growing stages of the vine varietal.FIGS. 2A and 2B are non-limiting examples. As those skilled in variouscrop growing arts would appreciate, there are many possible vinediseases and/or infestations that need to be managed for the variouscrops of various types/varietals.

Referring now to FIG. 3, wherein an example process for managing growingcrops with the assistance of autonomous vehicles, and components of anexample crop growing management system of FIG. 1, according to variousembodiments, are illustrated. As shown, for the illustrated embodiments,example crop growing management system 350 includes crop growingmanagement software 320 and CRSM 320. Crop growing management system 350may correspond to crop growing management system 110 of FIG. 1, and cropgrowing management software 320 may correspond to crop growingmanagement software of 120 of FIG. 1.

Crop growing management software 320 includes image processor 322,analyzer 324, reporter 326 and interactive report reader 328. Imageprocessor 322 is configured to process and combine/stitch togetherindividual aerial/overhead images taken in various spectrums to form aplurality of composite aerial/overhead images 332 of the crop growingfarm, e.g., a vineyard, in the various spectrums. Analyzer 324 isconfigured to machine process and analyze aerial/overhead images 302and/or terrestrial images 304 to identify anomalies with growing cropsof various types, e.g., vine of the various varietals in the varioussections of the crop growing farm, e.g., the vineyard. In someembodiments, analyzer 324 is configured to apply artificialintelligence, e.g., neural networks, to identify anomalies with growingcrops of various types, e.g., vine of the various varietals, in thevarious sections of the crop growing farm, e.g., the vineyard. Reporter326 is configured to machine generate one or more visual reports 336 ofthe crop growing farm, e.g., the vineyard, identifying/highlighting theanomalies associated with growing crops detected. In some embodiments,visual reports 336 may be two dimensional (2D) reports. In otherembodiments, visual reports 336 may be three dimensional (3D) reports orhalograms. Interactive report reader 328 is configured to facilitate auser in interactively viewing the visual report.

CRSM 330 is configured to store individual aerial/overhead images 302 ofvarious sections of the crop growing farm, e.g., a vineyard, captured invarious spectrums, as well as individual terrestrial images 304 ofvarious crops of various types, e.g., vines of various varietal,captured in the visual spectrum. CRSM 330 is also configured to storetopological information 306 of the crop growing farm, such as pond,creeks, streams and so forth, as well as current planting information308, i.e. crop types/varietals planted, and where. CRSM 330 may also beconfigured to store phenology information 310 of the croptypes/varietals, at different growing stages, and environmental data 312collected by local sensors and/or received from remote servers. CRSM 330may also be configured to store composite aerial/overhead images 332generated from individual aerial/overhead images 302, analysis results334 and reports 336.

In various embodiments, phenology information 310 of the croptypes/varietals may include various profiles of the various cropstypes/varietals over a growing season for different growing sections ofa crop growing farm. FIGS. 13a-13d illustrate various example profilesof various example crops types/varietals over example growing seasonsfor different growing sections of a number of example crop growingfarms. Specifically, FIG. 13a illustrates example NDVI profiles 1300 forvarious example apples (of the same or different types/varietals) overan example growing season from end of March to end of October forvarious sections of an example apple orchard. Each line represents anexample NDVI profile for the example apples grown in a section of theapple orchard. The vertical axis depicts the mean NDVI values, while thehorizontal axis depicts the dates in the growing season. Thealphanumeric identifiers of the profiles, shown in the bottom of theFigure, identify the sections of the example apple orchard.

FIG. 13b illustrates example NDVI profiles 1310 for various exampleblueberries (of the same or different types/varietals) over an examplegrowing season from end of March to end of October for various sectionsof an example blueberry orchard. Each line represents an example NDVIprofile for the example blueberries grown in a section of the blueberryorchard. The vertical axis depicts the mean NDVI values, while thehorizontal axis depicts the dates in the growing season. Thealphanumeric identifiers of the profiles, shown in the bottom of theFigure, identify the sections of the example blueberry orchard.

FIGS. 13c and 13d respectively illustrates example NDVI profiles 1320and 1330 for various example low productivity and high productivity hops(of the same or different types/varietals) over an example growingseason from end of March to end of October for various sections ofcouple of example hop growing farms. Each line represents an examplemean NDVI profile for the example hops grown in a section of an examplehop farm. The vertical axis depicts the mean NDVI values, while thehorizontal axis depicts the dates in the growing season. Thealphanumeric identifiers of the profiles, shown in the bottom of theFigures, identify the sections of the example hop farms.

FIG. 14 illustrates a number of example mean seasonal NDVI profiles 1400over an example growing season from mid-October to mid-March for anumber of example crops grown in a number of sites of an examplegeographical region. The vertical axis depicts the mean NDVI values,while the horizontal axis depicts the dates in the growing season. Theexample crops include pears, kiwis, cherries, avocados and mandarinoranges.

In other embodiments, the profiles may span different growing seasons,as well as other profiles, such as but not limited to NDRE profiles, maybe used instead or additionally.

In various embodiments, except for their usage CSRM 330 may be any oneof CRSM known in the art including, but are not limited to, non-volatileor persistent memory, magnetic or solid state disk drives, compact-diskread-only memory (CD-ROM), magnetic tape drives, and so forth.

Still referring to FIG. 3, process 300 for managing growing of crops,e.g., vines, include operations performed at stages A-E. Starting atstage A, the UAVs fitted with a plurality of cameras equipped tocapture/generate aerial images 302 in a plurality of spectrums areoperated, in succession or concurrently, to fly over all or selectedsections of the vineyard. And individual aerial images 302 of thevarious sections of the vineyard are taken as the UAVs fly over thesections at a selected altitude. As described, in some embodiments, theaerial images are taken at high resolution covering a small area of 3-4cm² per pixel or smaller. On capture/generation, individual aerialimages 302 of the various sections of the vineyard are stored into CRSM330. Optionally, overhead images of selected sections of the cropgrowing farm may be taken with imaging devices/cameras mounted onnettings and/or fixtures in the sections.

At stage A, the one or more terrestrial robots fitted with one or morecameras equipped to capture/generate terrestrial images 304 in visualspectrum may also be optionally operated, in succession or concurrently,to traverse all or selected sections of the crop growing farm, e.g., avineyard. And individual terrestrial images 304 of various crops ofvarious types or varietals being grown in the various sections of thecrop growing farm are taken as the terrestrial robots traverse over thesections. On capture/generation, individual terrestrial images 304 ofvarious crops of various types or varietals being grown in the varioussections of the crop growing farm are stored into CRSM 330.

From stage A, process 300 may proceed to stages B and C in parallel. Atstage B, image processor 322 may be executed on a computer system tomachine process and combine/stitch together individual aerial/overheadimages taken in various spectrums (take by the UAVs and/or stationarymounted cameras) to form a plurality of composite aerial/overhead images332 of the crop growing farm, in the various spectrums. On generation,composite aerial/overhead images 332 of the crop growing farm, in thevarious spectrums, are stored into CRSM 330.

At stage C, analyzer 324 may be executed on the computer system tomachine process individual aerial/overhead images 302 and/or individualterrestrial images 304 to machine analyze and detect anomaliesassociated with growing crops of the various types or varietals, takinginto consideration topological information of the crop growing farm, andcurrent planting information of the crop growing farm. Anomalies mayinclude, but are not limited, whether a section is under irrigated orover irrigated. By taking into consideration of the topologicalinformation, the analyzer may avoid false identification e.g., a pond ora creek as over irrigated, or a section growing crops of particulartypes, e.g., vines of a particular varietal, being stressed as underirrigated. In embodiments where terrestrial images 304 are also beingmachine processed and analyzed to detect anomalies associated withgrowing crops of the various types or varietals, the analysis may takeinto consideration phenology information of the various crops, atdifferent growing stages. Anomalies may include, but are not limited,various types of plant diseases and/or pest infestations.

In some embodiments, machine analysis of aerial/overhead images 302 aswell as terrestrial images 304, to detect anomalies associated withgrowing crops of various types or varietals may further take intoconsideration local environmental data 312 collected by local sensorsdisposed at various locations throughout the vineyard, and/orregional/areal environmental data 312 provided by one or more remoteenvironmental data services, applicable to the crop growing farm.

From stages B and C, process 300 may proceed to stage D. At stage D,reporter 326 may be executed on a computer system to machine generateone or more visual reports 336 of the crop growing farm, withindications of the anomalies detected. The visual reports 336 aregenerated using composite aerial/overhead images 332, and based at leastin part on the results of the analysis 334. As described earlier, visualreports 336 may be 2D, 3D or hologram.

Next, at stage E, on generation of visual reports 336, interactivereport reader 328 may be executed on a computing system to machinefacilitate interactive viewing of visual reports 336, by a user.

Still referring back to FIG. 3, in some embodiments, each of imageprocessor 322, analyzer 324, reporter 326 and interactive report reader328 may be implemented in hardware, software or combination thereof.Example hardware implementations may include by are not limited toapplication specific integrated circuit (ASIC) or programmable circuits(such as Field Programmable Gate Arrays (FPGA)) programmed with theoperational logic. Software implementations may include implementationsin instructions of instruction set architectures (ISA) supported by thetarget processors, or any one of a number of high level programminglanguages that can be compiled into instruction of the ISA of the targetprocessors. In some embodiments, especially those embodiments whereanalyzer includes at least one neural network, at least a portion ofanalyzer 324 may be implemented in an accelerator. One example softwarearchitecture and an example hardware computing platform will be furtherdescribed later with references to FIGS. 10 and 11.

Referring now to FIGS. 4A-4 b, wherein an example visual report toassist in managing crop growing, and an example user interface forinteractively viewing the visual report, according to variousembodiments, are illustrated. As shown in FIG. 4A and described earlier,example visual report 400 includes a composite image 402 of the cropgrowing farm, e.g., a vineyard, formed by combining or stitchingtogether the individual aerial images taken of the various sections ofthe crop growing farm, while the UAVs flew over the sections of the cropgrowing farm (and optionally, individual overhead images taken of thevarious sections of the crop growing farm by various stationary mountedimaging devices/cameras mounted on fixtures in the various sections).For the illustrated embodiments, the various sections of the cropgrowing farm may be annotated with various information 404 useful to theuser, including but are not limited to, section identificationinformation, crop types/varietals being grown, anomalies detected, andso forth. Example composite image 402 of an example vineyard, is formedby combining or stitching together the individual aerial/overhead NDREimages taken of the various sections of the example vineyard.

In various embodiments, visual report 400 is in colors. The differentgray scale levels in the drawing correspond to different colorsdepicting various conditions as captured by the aerial/overhead imagestaken in a particular spectrum, e.g., NDVI, NDRE and so forth.

For the illustrated embodiments, visual report 400 may further includevarious legend 406 and auxiliary information 408 to assist a user incomprehending the information provided. For example, legend 406 mayprovide the quantitative scale of a condition metric, such as soilmoisture level, corresponding to the different colors. Example auxiliaryinformation 408 may include, but are not limited to, time theaerial/overhead images are taken, air temperature at the time, windspeed at the time, wind direction at the time, and other observedconditions at the time.

FIG. 4B illustrates an example user interface for facilitating a user ininteractively viewing the visual reports with a user/client computingdevice. As shown, for the embodiments, user interface 450 includes anumber of tabs 452 a-452 c, one each for viewing a particular visualreport of aerial/overhead images taken in a particular spectrum. Eachtab, e.g., tab 452 a, includes main display area 454 for displaying thevisual report 462 annotated with highlights 464 of anomalies detected.Additional, each tab may further include section 456 for displayingvarious information, e.g., file information, and section 458 fordisplaying various command icons. In response to a selection of acommand, or a selection of an anomaly highlighted, a pop up window 466may be displayed to provide further information and/or facilitatefurther interaction.

Referring now to FIG. 5, wherein an example process for machineprocessing the aerial/overhead images, according to various embodiments,is illustrated. As shown, for the embodiments, example process 500 formachine processing/combining the aerial/overhead images to generate thecomposite aerial/overhead image include operations perform at blocks502-512. The operations may be performed by e.g., image processor 322 ofFIG. 3.

Example process 500 starts at block 502. At block 502, a corneraerial/overhead image may be retrieved. For example, it may be the upperleft corner image, the upper right corner image, the lower right cornerimage or the lower left corner image. Next at block 504, the next imagein the next column of the same row (or next row, same column) may beretrieved and combined/stitched with the previously processed images,depending on whether the aerial/overhead images are being combined on acolumn first basis (or a row first basis).

At block 506, a determination is made if the last column has beenreached, if the aerial/overhead images are being combed or stitched in acolumn first basis (or the last row has been reached, if theaerial/overhead images are being combed or stitched in a row firstbasis). If the last column of the row (or last row of the column) hasnot been reached, process 500 returns to block 504, and continuestherefrom as earlier described. If the last column of the row (or lastrow of the column) has been reached, process 500 proceeds to block 508.

At block 508, the next image in the first column of the next row, if theaerial/overhead images are being combed or stitched in a column firstbasis (or first row, next column, if the aerial/overhead images arebeing combed or stitched in a row first basis) may be retrieved andcombined/stitched with the previously processed images.

At block 510, a determination is made if the last row has been reached,if the aerial/overhead images are being combed or stitched in a columnfirst basis (or the last column has been reached, if the aerial/overheadimages are being combed or stitched in a row first basis). If the lastrow of the column (or last column of the row) has not been reached,process 500 proceeds to block 512.

At block 512, the next image in the next row, first column is retrieved,if the aerial/overhead images are being combed or stitched in a columnfirst basis (or next column, first row, if the aerial/overhead imagesare being combed or stitched in a row first basis), andcombined/stitched with the previously processed images. Thereafter,process 500 continues at block 504, as earlier described.

Eventually, it is determined at block 510 that the last row of thecolumn (or last column of the row) has been reached. At such time,process 500 proceeds to block 512.

While the combining/stitching process to generate the compositeaerial/overhead image has been described with an example process thatstarts at one of the 4 corners of a substantially rectangular sectionalpartition of the crop growing farm, it should be noted that the presentdisclosure is not so limited. The crop growing farm may be in any shape,and may be partitioned into sections in non-rectangular manner. Thecombining/stitching process may start with any aerial/overhead image,and radiates out to combine and stitch the next aerial/overhead image inany number of directions successively.

Referring now to FIG. 6, wherein an example process for machineanalyzing the images, according to various embodiments, is illustrated.As shown, for the embodiments, example process for machine analyzing theimages include operations at blocks 602-612. The operations at blocks602-612 may be machine performed by e.g., analyzer 324 of FIG. 3.

Process 600 starts at block 602. At block 602, an individualaerial/overhead image (and optionally, corresponding terrestrial imagesof the section) is (are) retrieved. Next, at block 604, topologicalinformation of the crop growing farm and current planting informationfor the section are retrieved. From block 604, process 600 may proceedto one of blocks 606, 608 or 610 depends on whether correspondingterrestrial images are also analyzed, and/or whether local/remoteenvironmental data are considered.

If corresponding terrestrial images are also analyzed, process 600proceeds to block 606. At block 606, phenology information of the croptypes/varietals being grown, at different stages, are retrieved for theanalysis. If environmental data are also being taken into consideration,process 600 also proceeds to block 608. At block 608, the local/remoteenvironmental data are retrieved.

From block 604, 606 or 608, process 600 eventually proceeds to block610. At block 610, the aerial/overhead image, and optionally,corresponding terrestrial images, are analyzed for anomalies associatedwith growing vine of the various varietals. As described earlier, theanalysis takes into consideration the topological information of thecrop growing farm, and current planting information. The analysis mayalso optionally take into consideration phenology information, atdifferent growing stages, and/or local/remote environmental data.

If anomalies are not detected, process 600 ends, otherwise the anomaliesare noted, before ending process 600. Process 600 may be repeated foreach section or selected sections of the crop growing farm.

Referring now to FIG. 7, wherein an example process for machinegenerating a visual report, according to various embodiments, isillustrated. As shown, for the embodiments, processor 700 includesvarious operations performed at blocks 702-710. The operations at blocks702-710 may be performed by e.g., reporter 326 of FIG. 3.

Example process 700 starts at block 702. At block 702, one or morecomposite aerial/overhead images are outputted. Next at block 704, anarea of the crop growing farm, e.g., a vineyard, is selected, and atblock 706, the selected area is examined to determine whether anomaliesassociated with growing crops of various types, such as vine of variousvarietals, were detected. If a result of the determination indicatesthat anomalies associated with growing crops of various types orvarietals were not detected, process 700 proceeds to block 710, elseprocess 700 proceeds to block 708, before proceeding to block 710. Atblock 708, the visual report is annotated to highlight the anomalydetected.

At block 710, a determination is made on whether there are additionalareas of the crop growing farm to be analyzed. If a result of thedetermination indicates there are additional areas of the crop growingfarm to be analyzed, process 700 returns to block 704, and proceedstherefrom as earlier described. Otherwise, process 700 ends.

Referring now FIG. 8, wherein an example process for machinefacilitating interactive viewing of a visual report, according tovarious embodiments, is illustrated. As shown, for the illustratedembodiments, process 800 for facilitating interactive viewing of thevisual report comprises operations at blocks 802-806. In someembodiments, operations at blocks 802-806 may be performed by e.g.,interactive report reader 328 of FIG. 3.

Process 800 may start at block 802. At block 802, a user request may bereceived. The user request may be associated with displaying a newvisual report, providing further information on a detected anomaly,providing remedial action suggestions for a detected anomaly, and soforth. Next at block 804, the user request may be processed. At block806, the processing results, e.g., the requested visual report, furtherexplanation of an anomaly of interest, a remedial action suggestion, andso forth, may be outputted/displayed for the user. The process resultsmay optionally further include facilities for further interaction by theuser.

Referring now to FIG. 9, wherein an example neural network, inaccordance with various embodiments, is illustrated. As shown, exampleneural network 900 may be suitable for use e.g., by analyzer 324 of FIG.3, in determining whether the images suggest anomalies associated withgrowing vines of the varietals. Example neural network 900 is amultilayer feedforward neural network (FNN) comprising an input layer912, one or more hidden layers 914 and an output layer 916. Input layer912 receives data of input variables (x) 902. Hidden layer(s) 914processes the inputs, and eventually, output layer 916 outputs thedeterminations or assessments (y) 904. In one example implementation theinput variables (x_(i)) 902 of the neural network are set as a vectorcontaining the relevant variable data, while the output determination orassessment (y) 904 of the neural network are also as a vector.

Multilayer feedforward neural network (FNN) may be expressed through thefollowing equations:

${{ho_{i}} = {f\left( {{\sum\limits_{j = 1}^{R}\left( {iw_{i,j}x_{j}} \right)} + {hb_{i}}} \right)}},{{{for}\mspace{14mu} i} = 1},\ldots\mspace{14mu},N$${y_{i} = {f\left( {{\sum\limits_{k = 1}^{N}\left( {hw_{i,k}ho_{k}} \right)} + {ob_{i}}} \right)}},{{{for}\mspace{14mu} i} = 1},\ldots\mspace{14mu},S$

where ho_(i) and y_(i) are the hidden layer variables and the finaloutputs, respectively. f( ) is typically a non-linear function, such asthe sigmoid function or rectified linear (ReLu) function that mimics theneurons of the human brain. R is the number of inputs. N is the size ofthe hidden layer, or the number of neurons. S is the number of theoutputs.

The goal of the FNN is to minimize an error function E between thenetwork outputs and the desired targets, by adapting the networkvariables iw, hw, hb, and ob, via training, as follows:

${E = {\sum\limits_{k = 1}^{m}\left( E_{k} \right)}},{{{where}\mspace{14mu} E_{k}} = {\sum\limits_{p = 1}^{S}\left( {t_{kp} - y_{kp}} \right)^{2}}}$

where y_(kp) and t_(kp) are the predicted and the target values of pthoutput unit for sample k, respectively, and m is the number of samples.

In some embodiments, analyzer 324 of FIG. 3 may include a pre-trainedneural network 900 to determine whether an aerial/overhead and/or aterrestrial image suggests one or more anomalies associated with growingvines of the varietals. The input variables (x) 902 may include anaerial/overhead image of a section captured in a particular spectrum,one or more corresponding terrestrial images, topological data, currentplanting data, phenology data, and environmental data. The outputvariables (y_(i)) 904 may include values indicating whether anomaliesare detected and/or anomaly types. The network variables of the hiddenlayer(s) for the neural network of analyzer 324 for determining whetheran aerial/overhead image of a section captured in a particular spectrumand/or corresponding terrestrial images suggest anomalies with the vinesbeing grown, are determined by the training data.

In the example neural network of FIG. 9, for simplicity of illustration,there is only one hidden layer in the neural network. In some otherembodiments, there can be many hidden layers. Furthermore, the neuralnetwork can be in some other types of topology, such as ConvolutionNeural Network (CNN), Recurrent Neural Network (RNN), and so forth.

Referring now to FIG. 10, wherein a software component view of the cropgrowing management (CGM) system, according to various embodiments, isillustrated. As shown, for the embodiments, V\CGM system 1000, whichcould be CGM system 110, includes hardware 1002 and software 1010.Software 1010 includes hypervisor 1012 hosting a number of virtualmachines (VMs) 1022-1028. Hypervisor 1012 is configured to hostexecution of VMs 1022-1028. The VMs 1022-1028 include a service VM 1022and a number of user VMs 1024-1028. Service machine 1022 includes aservice OS hosting execution of a number of system services andutilities. User VMs 1024-1028 may include a first user VM 1024 having afirst user OS hosting execution of image processing 1034, a second userVM 1026 having a second user OS hosting execution of analysis and reportgeneration 1036, and a third user VM 1028 having a third user OS hostingexecution of interactive report viewing, and so forth.

Except for the crop growing management technology of the presentdisclosure incorporated, elements 1012-1038 of software 1010 may be anyone of a number of these elements known in the art. For example,hypervisor 1012 may be any one of a number of hypervisors known in theart, such as KVM, an open source hypervisor, Xen, available from CitrixInc, of Fort Lauderdale, Fla., or VMware, available from VMware Inc ofPalo Alto, Calif., and so forth. Similarly, service OS of service VM1022 and user OS of user VMs 1024-1028 may be any one of a number of OSknown in the art, such as Linux, available e.g., from Red Hat Enterpriseof Raliegh, N.C., or Android, available from Google of Mountain View,Calif.

In alternate embodiments, where CGM 1000 may be configured to servicemultiple crop growing farms, each user VM 1024, 1026 and 1028 may beconfigured to respectively handle image processing, analysis, reportgeneration, and interactive report viewing of one crop growing farm, toprovide data isolation between crop growing farms.

Referring now to FIG. 11, wherein an example computing platform that maybe suitable for use to practice the present disclosure, according tovarious embodiments, is illustrated. As shown, computing platform 1100,which may be hardware 1002 of FIG. 10, may include one or moresystem-on-chips (SoCs) 1102, ROM 1103 and system memory 1104. Each SoCs1102 may include one or more processor cores (CPUs), one or moregraphics processor units (GPUs), one or more accelerators, such ascomputer vision (CV) and/or deep learning (DL) accelerators. ROM 1103may include basic input/output system services (BIOS) 1105. CPUs, GPUs,and CV/DL accelerators may be any one of a number of these elementsknown in the art. Similarly, ROM 1103 and BIOS 1105 may be any one of anumber of ROM and BIOS known in the art, and system memory 1104 may beany one of a number of volatile storage known in the art.

Additionally, computing platform 1100 may include persistent storagedevices 1106. Example of persistent storage devices 1106 may include,but are not limited to, flash drives, hard drives, compact discread-only memory (CD-ROM) and so forth. Further, computing platform 1100may include input/output (I/O) device interface to couple I/O devices1108 (such as display, keyboard, cursor control and so forth) to system1100, and communication interfaces 1110 (such as network interfacecards, modems and so forth). Communication and I/O devices 1108 mayinclude any number of communication and I/O devices known in the art.I/O devices may include in particular sensors 1120, which may be some ofthe sensors 108 of FIG. 1. Examples of communication devices mayinclude, but are not limited to, networking interfaces for Bluetooth®,Near Field Communication (NFC), WiFi, Cellular communication (such asLTE 4G/5G) and so forth. The elements may be coupled to each other viasystem bus 1112, which may represent one or more buses. In the case ofmultiple buses, they may be bridged by one or more bus bridges (notshown).

Each of these elements may perform its conventional functions known inthe art. In particular, ROM 1103 may include BIOS 1105 having a bootloader. System memory 1104 and mass storage devices 1106 may be employedto store a working copy and a permanent copy of the programminginstructions implementing the operations associated with hypervisor 112,service/user OS of service/user VM 1022-1028, and components of the CGMtechnology (such as image processor 322, analyzer 324, reporter 326 andinteractive report reader 328, and so forth), collectively referred toas computational logic 922. The various elements may be implemented byassembler instructions supported by processor core(s) of SoCs 1102 orhigh-level languages, such as, for example, C, that can be compiled intosuch instructions.

As will be appreciated by one skilled in the art, the present disclosuremay be embodied as methods or computer program products. Accordingly,the present disclosure, in addition to being embodied in hardware asearlier described, may take the form of an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to as a “circuit,” “module” or “system.”Furthermore, the present disclosure may take the form of a computerprogram product embodied in any tangible or non-transitory medium ofexpression having computer-usable program code embodied in the medium.FIG. 12 illustrates an example computer-readable non-transitory storagemedium that may be suitable for use to store instructions that cause anapparatus, in response to execution of the instructions by theapparatus, to practice selected aspects of the present disclosure. Asshown, non-transitory computer-readable storage medium 1202 may includea number of programming instructions 1204. Programming instructions 1204may be configured to enable a device, e.g., computing platform 1100, inresponse to execution of the programming instructions, to implement(aspects of) hypervisor 112, service/user OS of service/user VM 122-128,and components of CGM technology (such as mage processor 322, analyzer324, reporter 326 and interactive report reader 328, and so forth) Inalternate embodiments, programming instructions 1204 may be disposed onmultiple computer-readable non-transitory storage media 1202 instead. Instill other embodiments, programming instructions 1204 may be disposedon computer-readable transitory storage media 1202, such as, signals.

Any combination of one or more computer usable or computer readablemedium(s) may be utilized. The computer-usable or computer-readablemedium may be, for example but not limited to, an electronic, magnetic,optical, electromagnetic, infrared, or semiconductor system, apparatus,device, or propagation medium. More specific examples (a non-exhaustivelist) of the computer-readable medium would include the following: anelectrical connection having one or more wires, a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CD-ROM), an optical storage device, a transmission media such as thosesupporting the Internet or an intranet, or a magnetic storage device.Note that the computer-usable or computer-readable medium could even bepaper or another suitable medium upon which the program is printed, asthe program can be electronically captured, via, for instance, opticalscanning of the paper or other medium, then compiled, interpreted, orotherwise processed in a suitable manner, if necessary, and then storedin a computer memory. In the context of this document, a computer-usableor computer-readable medium may be any medium that can contain, store,communicate, propagate, or transport the program for use by or inconnection with the instruction execution system, apparatus, or device.The computer-usable medium may include a propagated data signal with thecomputer-usable program code embodied therewith, either in baseband oras part of a carrier wave. The computer usable program code may betransmitted using any appropriate medium, including but not limited towireless, wireline, optical fiber cable, RF, etc.

Computer program code for carrying out operations of the presentdisclosure may be written in any combination of one or more programminglanguages, including an object oriented programming language such asJava, Smalltalk, C++ or the like and conventional procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The program code may execute entirely on the user's computer,partly on the user's computer, as a stand-alone software package, partlyon the user's computer and partly on a remote computer or entirely onthe remote computer or server. In the latter scenario, the remotecomputer may be connected to the user's computer through any type ofnetwork, including a local area network (LAN) or a wide area network(WAN), or the connection may be made to an external computer (forexample, through the Internet using an Internet Service Provider).

The present disclosure is described with reference to flowchartillustrations and/or block diagrams of methods, apparatus (systems) andcomputer program products according to embodiments of the disclosure. Itwill be understood that each block of the flowchart illustrations and/orblock diagrams, and combinations of blocks in the flowchartillustrations and/or block diagrams, can be implemented by computerprogram instructions. These computer program instructions may beprovided to a processor of a general purpose computer, special purposecomputer, or other programmable data processing apparatus to produce amachine, such that the instructions, which execute via the processor ofthe computer or other programmable data processing apparatus, createmeans for implementing the functions/acts specified in the flowchartand/or block diagram block or blocks.

These computer program instructions may also be stored in acomputer-readable medium that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablemedium produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide processes for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a,” “an” and “the” are intended toinclude plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specific thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operation, elements,components, and/or groups thereof.

Embodiments may be implemented as a computer process, a computing systemor as an article of manufacture such as a computer program product ofcomputer readable media. The computer program product may be a computerstorage medium readable by a computer system and encoding a computerprogram instructions for executing a computer process.

The corresponding structures, material, acts, and equivalents of allmeans or steps plus function elements in the claims below are intendedto include any structure, material or act for performing the function incombination with other claimed elements are specifically claimed. Thedescription of the present disclosure has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the disclosure in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill without departingfrom the scope and spirit of the disclosure. The embodiment was chosenand described in order to best explain the principles of the disclosureand the practical application, and to enable others of ordinary skill inthe art to understand the disclosure for embodiments with variousmodifications as are suited to the particular use contemplated.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the disclosed embodiments ofthe disclosed device and associated methods without departing from thespirit or scope of the disclosure. Thus, it is intended that the presentdisclosure covers the modifications and variations of the embodimentsdisclosed above provided that the modifications and variations comewithin the scope of any claims and their equivalents.

What is claimed is:
 1. A method for managing growing crops in a cropgrowing farm, comprising: operating one or more unmanned aerial vehicles(UAVs) to fly over a plurality of sections of the crop growing farm, theone or more unmanned UAVs together being fitted with a plurality ofcameras equipped to generate images in a plurality of spectrums, whereinthe plurality of sections of the crop growing farm grow crops of one ormore types or varietals, and the plurality of cameras include aRed-Green-Blue (RGB) camera equipped to generate images in a visualspectrum, a Normalized Difference Vegetation Index (NDVI) cameraequipped to generate images in red or near infrared spectrum, and aNormalized Difference Red Edge (NDRE) camera equipped to generate imagesin near red spectrum; taking a plurality of aerial images of thesections of the crop growing farm in the plurality of spectrums, usingthe plurality of cameras, while the one or more UAVs are flying over theplurality of sections of the crop growing farm; storing the plurality ofaerial images in a computer readable storage medium (CRSM); executing ananalyzer on a computing system with access to the CRSM to machineanalyze the plurality of aerial images in the visual, red or nearinfrared, and near red spectrums, and a plurality of overhead orterrestrial images for anomalies associated with growing the crops ofthe one or more types or varietals in the plurality of sections of thecrop growing farm, including potential pest infestations, the machineanalysis taking into consideration topological information of the cropgrowing farm, as well as current planting information of the cropgrowing farm; and executing a reporter on the computing system tomachine produce a visual report with indications of crop growinganomalies in the plurality of sections of the crop growing farm,including potential pest infestations, to assist in managing growing thecrops of one or more types or varietals in the plurality of sections ofthe crop growing farm, the indication of crop growing anomalies,including potential pest infestations, being based at least in part onresults of the analysis of the visual, red or near infrared, and nearred spectrum images, and the terrestrial images.
 2. The method of claim1, wherein operating the one or more UAVs to fly over the plurality ofsections of the crop growing farm comprises operating one or more fixedwing or lightweight UAVs to systematically fly over all sections of thecrop growing farm.
 3. The method of claim 1, wherein operating the oneor more UAVs to fly over the plurality of sections of the crop growingfarm comprises operating a UAV having the RGB, NDVI and NDRE cameras togenerate the visual, red or near infrared, and near red spectrum images.4. The method of claim 1, wherein the crop growing farm spans over 100acres, with over 1000 plants planted per acre, and each pixel of each ofthe plurality of aerial images covers about 3-4 cm² or smaller.
 5. Themethod of claim 1, wherein to machine analyze the plurality of aerialimages and a plurality of overhead or terrestrial images comprises tomachine identify one or more of the sections as potentially in an overirrigated state or an under irrigated state for the crop types orvarietals being grown in the one or more sections, based at least inpart on the topological information of the crop growing farm, as well asthe current planting information of the crop growing farm.
 6. The methodof claim 5, wherein executing the reporter on the computing systemcomprises executing the reporter on the computing system to machineproduce the visual report with indications of the one or more sectionsidentified as potentially in an over irrigated state or an underirrigated state for the crop types or varietals being grown in the oneor more sections.
 7. The method of claim 1, further comprising executingan image processing program on the computing system to machine processthe plurality of aerial images to combine the plurality of aerial imagesto produce one or more composite images of the crop growing farm;wherein the reporter machine produces the visual report using the one ormore composite images of the crop growing farm.
 8. The method of claim1, further comprising: operating one or more terrestrial robots totraverse the plurality of sections of the crop growing farm, the one ormore terrestrial robots being fitted with one or more cameras equippedto generate the plurality of terrestrial images in the visual spectrum;and taking the plurality of visual spectrum terrestrial images of thecrops of the one or more types or varietals being grown in the pluralityof sections of the crop growing farm, using the one or more camerasfitted on the one or more terrestrial robots, while the one or moreterrestrial robots are traversing the plurality of sections of the cropgrowing farm.
 9. The method of claim 8, further comprising storing theplurality of visual spectrum terrestrial images of the crops of the oneor more types or varietals being grown, in the CRSM; and whereinexecuting the analyzer further comprises executing the analyzer on thecomputing system to machine analyze the plurality of visual spectrumterrestrial images, with the machine analysis of the visual spectrumterrestrial images taking into consideration phenology information ofthe crop types or varietals being grown, at different growing stages.10. The method of claim 9, wherein to machine analyze the plurality ofvisual spectrum terrestrial images for additional anomalies comprises tomachine identify potential pest infestation in one or more of the cropsof the one or more types or varietals being grown in the one or more ofthe sections of the crop growing farm, based at least in part on thephenology information of the crop types or varietals being grown, atdifferent growing stages.
 11. The method of claim 10, wherein executingthe reporter on the computing system comprises executing the reporter onthe computing system to machine produce the visual report withindications of the one or more crops of the crops of one or more typesor varietals being grown in the one or more of the sections of the cropgrowing farm as being potentially pest infested.
 12. The method of claim1, further comprising sensing with a plurality of sensors disposed in atleast selected ones of the plurality of sections of the crop growingfarm environmental data at various locations in the selected ones of thesections of the crop growing farm; and wherein machine processing andanalyzing of the plurality of aerial and overhead or terrestrial imagesis further in view of the environmental data sensed at the variouslocations in the selected ones of the sections of the crop growing farm.13. The method of claim 1, further comprising receiving, from anexternal source, environmental data applicable to the crop growing farm;and wherein machine processing and analyzing of the plurality of aerialand overhead or terrestrial images is further in view of theenvironmental data applicable to the crop growing farm.
 14. The methodof claim 1, further comprising executing an interactive report reader onthe computing system to facilitate a user in viewing the visual reportinteractively.
 15. The method of claim 1, wherein the crop is a selectedone of hops, kiwis, apples, berries, cherries, citrus fruit, avocado,pears, plums, hemp, or cannabis, or the crop growing farm is a selectedone of a fruit growing farm or a vegetable growing farm.
 16. A methodfor managing growing crops in a crop growing farm, comprising: operatingone or more terrestrial robots to traverse a plurality of sections ofthe crop growing farm, the one or more terrestrial robots being fittedwith one or more cameras equipped to generate images in visual spectrum,and the plurality of sections of the crop growing farm growing crops ofone or more types or varietals; taking a plurality of visual spectrumterrestrial images of the crops of the one or more types or varietalsgrowing in the sections of the crop growing farm, using the one or morecameras, while the one or more terrestrial robots are traversing theplurality of sections of the crop growing farm; storing the plurality ofvisual spectrum terrestrial images in a computer readable storage medium(CRSM); executing an analyzer on a computing system with access to theCRSM to machine analyze the plurality of visual spectrum terrestrialimages and a plurality of aerial or overhead images for anomaliesassociated with growing the crops of the one or more types or varietalsin the plurality of sections of the crop growing farm, where theplurality of aerial or overhead images include visual, red or nearinfrared, and near red images respectively taken by a Red-Green-Blue(RGB) camera, a Normalized Difference Vegetation Index (NDVI) camera,and a Normalized Difference Red Edge (NDRE) camera, the anomaliesincluding pest infestation among the crops, and the machine analysistaking into consideration current planting information of the cropgrowing farm and phenology information of the crops of the one or moretypes or varietal being grown in the plurality of sections of the cropgrowing farm, at different growing stages; and executing a reporter onthe computing system to machine produce a visual report with indicationsof growing anomalies of the crops to assist in managing growing thecrops of one or more types or varietals in the plurality of sections ofthe crop growing farm, the indication of growing anomalies being basedat least in part on results of the analysis.
 17. The method of claim 16,wherein executing the analyzer comprises executing the analyzer on thecomputing system to process and analyze the plurality of visual spectrumterrestrial images in view of the current planting information of thecrop growing farm and the phenology information of the crops of the oneor more types or varietals being grown at different growing stages, toidentify one or more crops being grown in one or more of the sections aspotentially being infested.
 18. The method of claim 17, whereinexecuting the reporter comprises executing the reporter on the computingsystem to machine produce the visual report with highlights of theidentified potential pest infestations.
 19. The method of claim 16,wherein the crop is a selected one of hops, kiwis, apples, berries,cherries, citrus fruit, avocado, pears, plums, hemp, cannabis, or thecrop growing farm is a selected one of a fruit growing farm or avegetable growing farm.
 20. At least one non-transitory computerreadable medium (CRM) having instructions stored therein to cause acomputing system, in response to execution of the instructions by thecomputing system, to: analyze a plurality of aerial or overhead imagesin a plurality of spectrums and a plurality of terrestrial images foranomalies associated with growing crops of one or more types orvarietals in a plurality of sections of a crop growing farm, taking intoconsideration topological information of the crop growing farm, as wellas current planting information of the crop growing farm, the pluralityof aerial or overhead images include visual, red or near infrared, andnear red images respectively taken by an airborne Red-Green-Blue (RGB)camera, an airborne Normalized Difference Vegetation Index (NDVI)camera, and an airborne Normalized Difference Red Edge (NDRE) camera,the plurality of terrestrial images being captured by one or more rovingcameras, and the anomalies being analyzed include pest infestations; andproduce a visual report with indications of crop growing anomalies inthe plurality of sections of the crop growing farm to assist in managinggrowing the crops in the crop growing farm, the indication of cropgrowing anomalies, including pest infestations, being based at least inpart on results of the analysis; wherein the plurality of aerial imagesof the sections of the crop growing farm in the plurality of spectrumsare taken using the RGB, NDVI and NDRE cameras fitted on one or moreunmanned aerial vehicles (UAVs), while the one or more UAVs are flyingover the plurality of sections of the crop growing farm.
 21. The CRM ofclaim 20, wherein to analyze the plurality of aerial or overhead imagescomprises to identify one or more of the sections as potentially in anover irrigated state or an under irrigated state for the crops of theone or more types or varietals being grown in the one or more sections,based at least in part on the topological information of the cropgrowing farm, as well as the current planting information of the cropgrowing farm.
 22. The CRM of claim 21, wherein to produce the visualreport comprises to produce the visual report with indications of theone or more sections identified as potentially in the over irrigatedstate or the under irrigated state for the crops of the one or moretypes or varietals being grown in the one or more sections.
 23. The CRMof claim 20, wherein the analysis of the visual spectrum terrestrialimages takes into consideration phenology information of the crops ofone or more types or varietals being grown, at different growing stages.24. The CRM of claim 23, wherein to identify potential pest infestationsis based at least in part on the phenology information of the crops ofthe one or more types or varietals being grown, at different growingstages.
 25. The CRM of claim 24, wherein to produce the visual reportcomprises to produce the visual report with indications of one or morecrops of the one or more types or varietals being grown in the one ormore of the sections of the crop growing farm as being potentially pestinfested.